Module 13 - Deep Learning

Overview

During this week, we continue to learn about neural networks, moving from the basics of individual neurons to challenges that arise in deep architectures. We introduce recurrent neural networks, deep neural networks, and discuss some of the challenges with training deep neural networks. We explain backpropagation, and show how to use ReLU layers to mitigate vanishing and/or exploding gradients, and talk about hyperparameter optimization in stochastic gradient descent.

Lab 7 is due at the end of the week.

Learning Objectives

  • RNN Architectures
  • Challenges with Deep Learning: Vanishing and Exploding Gradients
  • ReLU Layers
  • Hyperparameter Choices for Stochastic Gradient Descent

Readings

  • ISLP (Introduction to Statistical Learning): 10.5-10.8

Videos